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The canonical duality theory has provided with a unified analytic solution 
to a range of discrete and continuous problems in global optimization, which 
can transform a nonconvex primal problem to a concave maximization dual 
problem over a convex domain without duality gap. This paper shows that 
under certain conditions, this canonical dual problem is equivalent to the stan- 
dard semi-definite programming (SDP) problem, which can be solved by well- 
developed software packages. In order to avoid certain difficulties of using the 
SDP method, four strategies are proposed based on unconstrained approaches, 
which can be used to develop algorithms for solving some challenging problems. 
Applications are illustrated by fourth-order polynomials benchmark optimiza- 
tion problems. 



Keywords: Global optimization; Optimization algorithms; Canonical duality 
theory; 



Introduction and motivation 



Numerical optimization methods are usually categorized into deterministic and 
stochastic, both of them have found extensive ap plications in real-world problems 
( Hendrix and Toth 2010 . Shmovs and Swamvl 2004 ). The deterministic methods such 
as Newton's method were considered as "local search" because they are dependent 
on initial point to a large extent and finally arrive at the "neighborhood" of the 
initial point, while the stochastic methods such as genetic algorithm were regarded 
as "global search" due to their search ability on the whole space. However, the 
premature convergence and easily get ting trapped into local optimum are common 



phenomena for stochastic algorithms (|Back et all 119971 . iBentley et all l200ll ). On the 



other hand, some deterministic global optimization algorithms have been proposed 
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in recent years, and they are able to solve much general optimization problems 
such as nonconvex c ontinuous, mixed-integer, dif ferential-algebraic, bilevel, and non- 
factorable problems ( Floudas and Gounaris 20091 ) . The development of deterministic 
and stochastic just indicate what the "No Free Lunch Theorems" mean s, that is, ther e 
exists no algorithm which is better than its competitor over all problems ( Wolpert 19971 ). 



In the meanwhile, a novel global optimization theory called the canonical duality 
theory has been developed during the past 20 years. The kernels of the theory consist 
of a canonical du al transformation methodology, a complementary-dual principle, and 
a triality theory ( Gaol 200d . 120091 ). The main merit is that this theory can transform 
nonconvex/nonsmoonth/discrete optimization/variational problems into continuous 
concave maximization problems over convex domains, which can be solved easily, 
under certain conditions, by many well-developed algorithms and softwares. Therefore, 
the canonical duality theory has been used suc cessfully for solvin g a large class of 

), en gineering me- 
2011), information 



challengi ng problems in computational biology (IZhang et al\ 12 01 1 



chanics (Ga o and Sherali 
theory ( ILatorre and Gac 



2009 



Gao and Yu 2008 



Santos and Gao 



2012 ), network com munications ( Gao et al. 2012al ). nonlin 



ear dynamica l systems |R~and GacJB. and some NPl u ^obleW in global 



optimization ( Fang et al. 20081 . Gao and R,uanll2010l . iGao et al 1 l2012bl . lWang et fli.ll2012l ) . 



However, it was realized that the canonical dual problem may have no critical point 
in th e dual feasible space and in this case, the primal problem could be NP-hard (jGaol 
20071 ). By introducing a linear perturbation term to the primal problem or a quadratic 
perturbation term to the dual proble m, the issue can b e partially tackled with to some 
extent but is still an open problem (jWang et aZ.I |2012| ) . For one thing, it is not easy 



to find such an appropriate perturbation. For another, only approximate solution is 
obtained due to the perturbation. On the other hand, it is undoubtedly that solving 
a constrained optimization problem (a continuous concave maximization problem 
over convex domain) is much more difficult than an unconstrained one. Therefore, 
some approaches, such as the penalty function method, aim to convert a constrained 
minimization problem into an equivalent unconstrained one to reduce the computational 
complexity. 



As is known to us, for nonconvex optimization problem, methods based on gradient are 
dependent on initial point, and choosing a good initial point can reach a good solution 
in the end. To overcome local optimality, it usually requires some type of diversification 
to find the global optimum. For instance, the multi-start methods, which are applied 
by starti ng from multiple random initial solutions, are widely used to realize diver- 
sification (jMarti et a/l2009l ). However, it remains hard to construct good initial solutions. 



Fortunately, the global optimality condition contained in triality theory, can identify 
the global minimum, which provides with greatly useful information to select a good 
initial point and can be utilized to develop related canonical dual algorithms. In this 
study, we show that, under certain conditions, the canonical dual problem is essentially 
equivalent to the standard semi-definite programming ( SDP) probl em and then be solved 
by well-developed software packages, such as SeDuMi (|Sturnl ll999). In the case that the 
canonical dual problem has no critical point, four strategies are proposed to develop effi- 
cient algorithms based on unconstrained approaches by using the core points of canonical 
dual transformation methodology, complementary-dual principle and global optimality 
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condition. A series of fourth-order polynomials benchmark optimization problems are 
provided to demonstrate the effectiveness and efficiency of the proposed strategies. 



2. A brief review of canonical duality theory 

For the completeness of this paper, we give a brief revi ew of the followin g fourth-order 



polynomials minimization problem (primal problem) in Gao et al. ( 2012al ): 



(V) : minjp(x) = W{x) + ^x T Qx - x T f : x G M n J, (1) 

where, 

m \ \ 2 

W(x) = ^-a fc (-x T ^x + b)[x + c fc ) , (2) 

k=l 

and Ak = A^,Q = Q T G M nxn are indefinite symmetrical matrices, bfc,f € W 1 are 
given vectors, ak,c k € M are known constants. Without loss of much generality, the a k 
is assumed to be positive. 

The standard canonical dual transformation methodology consists of the following 
three procedures. 



2.1. Canonical dual transformation 

Introducing a nonlinear operator(a Gateaux differentiable geometrical measure) 

£ = (Ci, • • • ,Uf = A(x) = [\x T A k x + b£x + c k y :R n ^£ a cR m (3) 
so that W(x) can be recast by: 

W(x) = F(A(x)), (4) 
where, V(£) is said to be a canonical function and in this case 

- 1 1 

V(t) = T,2 ak & = 2 qT( ^°°' (5) 

k=l 

in which, a. = (ax,'"" ;«m) T > the notation sot = (sis\,--- ,s m s m ) T denotes the 
Hadamard product for any two vectors s, t € M m . 

Then, the primal problem can be rewritten as the canonical form: 



mm 



n{p(x) = y(A(x))-^(x)}, (6) 
where U(x) = — ix T Qx + x T f. 
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2.2. Generalized complementary function 

The dual variable ? to ^ is defined by the duality mapping 

? = (?!,-•■ ,Sm) = VV(H) = cx o £ : £ a ^ £* a C R m . (7) 

For the given canonical function V(£), the Legendre conjugate V*(q) can be defined 
by: 

= sta{^ - = £ -aj^, (8) 

5 k=i 

where, sta{-} stands for finding stationary point of the statement in {•}. The (£, <j) forms 
a canonical duality pair and the following canonical duality relations hold on £ a x £*: 

? = w(£) ^ c = w*(<0 o v(t) + = (9) 

Replacing W(x) = F(A(x)) by A T (x)<j — F*(c), the generalized complementary func- 
tion can be defined by 

H(x,<j) = A T (xK-y*W-f/(x) 

m 1 11 

= E [(2 xT ^ fcX + b ^ X + ° k ) qk ~ 2^ A + 2 XTQX " xTf - (10) 
k=l 



2.3. Canonical dual function 

By using the generalized complementary function, the canonical dual function P d (s) can 
be formulated as 

P d (0 = sta{£(x,<;)}. (11) 

For a fixed the stationary condition VH(x,c) leads to the canonical equilibrium 
equation: 

GWx = %), (12) 

in which, = Q + ZT=i^A k , = f - £^ =1 ftb fc . For any given <j, if is 

in the column space of G(q), denoted by C i{G{qj), i.e., a linear space spanned by the 
columns of G(q), the solution of the canonical equilibrium equation can be well defined 
by 

x = G*(?)F(*), (13) 

where, G'(<») denotes the Moore- Penrose generalized inverse of 
Then, the canonical dual function can be written explicitly as follows 

fe=i 
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Finally, the canonical dual problem can be expressed by 

(V d ) : sta{p d (<;) = jr [c k , k - l -a k \£) - \f t {<;)&{<;) F{<;) : « G S a }, (15) 
fc=i 

where the dual feasible space is defined by S a = {<? G R m |F(<j) G C ^(G(?))}. 

Theorem 1 (Complementary-Dual Principle and Analytical Solution). The 

problem (P d ) is canonically dual to the primal problem (V) in the sense that if q is 
a critical point of (V d ), then the vector 

x = C%)F(c) (16) 

is a critical point of (V) and 

P(x) = P d («;). (17) 

This theorem shows that the critical solutions to the primal problem depend analyt- 
ically on the canonical dual solutions and there is no duality gap between the primal 
problem and its canonical dual. 

Theorem 2 (Global Optimality Condition). Suppose q is a critical point of P d (<j). If 
<? G S£, then ^ is a global maximizer of (V d ) on 5+ if and only if the analytical solution 
x = G^(s)F(s) is a global minimizer of (V) on IR n , i.e., 

P(x) = min P(x) & max P d (<j) = P d (?), (18) 
x€R" ?e ,s+ 

where 

5+ = {, G 5 a |G(?) b 0}. (19) 

This theorem shows that <j G provides a global optimality condition, which can be 
used to develop algorithms for solving the nonconvex primal problem. 



3. The equivalent semi-definite programming problem 

By Theorem 2, the primal problem is equivalent to the following canonical dual maxi- 
mization problem ((P max ) in short): 

(P max ) : maxP d ( ? ) = £ (c fcft - -a fc A fc 2 ) - -P^)^^) (20) 

k=l 

In this section, we will show that this problem can be also equivalent to the standard 
semi-definite programming problem (SDP). 
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(SDP) : min -t x + -t 2 - <? T c 

subject to: (^^^O, 

(Diag{ai, ■ ■ ■ ,a m } <; 



h 0. (21) 



Theorem 3 Let (<;,ti,t 2 ) be an optimal solution of problem (SDP), if G(<;) >- 0, then <j 
is the unique optimal solution of problem (V d ) and x = G^(^)F(q) is the unique optimal 
solution of problem (V). If det(G(?)) = and (I - G(s)Gft(s))F(q) = 0, then ? is an 
optimal solution of problem iV d ) and x = \^)F(q) is an optimal solution of problem 
(V). In this case, problem (V) has multiple optimal solutions. 

Proof. At first, we relax (V d lSLX ) to the following form 

1 1 rp 

mm -ti + -t 2 -<; c 

subject to : t± > F T (<j)G _1 (<?)F(^), 

t 2 > q T Diag{a 1 , ■■■ , a m }<;, 

G{a) y 0, (22) 

where, c = [c\, . . . , c m ] T ', Diag{a\, • • • , a m } stands for a diagonal matrix with «i, • • • , a m 
as its elements. 

Lemma 1 (Schur complement) Considering the partitioned symmetric matrix 

x = xT = c) • < 23 > 

if ^ >- 0, then X ^ if and only if the matrix C - B T A~ 1 B y 0. 

Using the Schur complement lemma, we can get the equivalent positive (semi) definite 
programming optimization problem (SDP) consequently according to Theorem 1 and 
Theorem 2. □ 



4. Four strategies for canonical dual theory 

Although the canonical dual problem can be transformed into the equivalent semi-definite 
programming problem and then solved by well-developed software packages, it should be 
noted that there may be no critical points in the canonical dual feasible domain. More- 
over, solving a constrained optimization problem (SDP) is more complicated than an 
unconstrained one. In this paper, we focus on unconstrained methods, trying to explore 
efficient and effective algorithms based on the canonical duality theory. 

The canonical duality theory has provided with a unified analytic solution for opti- 
mization problems. Obviously, we can firstly find all of the stationary points, and then 
identify which one is in the canonical dual feasible domain <S+. As can be seen from 
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the main procedures of canonical dual transformation methodology, we can find that we 
have to solve stationary problems twice, one is for S(x, <;), and the other is for P d (<;). 
To calculate the stationary points for S(x, we have to solve the following nonlinear 
equations: 

(GWx = F(?), 

{ 1 ( 24 ) 
[ 2 xT ^x + x + c fe = a k l q k ,Vk = 1, • • • ,m. 

Similarly, to find the stationary points for P d (<;), we have to solve the nonlinear equations 
as follows: 

\F T ( q )G^A k G\<;)F{<;) + b£G%)F(?) + c k - a~\ k = 0, Vfe = 1, • • • ,m. (25) 

As the number of variables becomes large, the complexity of computing the stationary 
problems is also increasing. That is to say, the computing of the stationary points for 
S(x, <;) will be more complicated than that of P d (<j). On the contrary, we can also observe 
that the solving of stationary points for P d (<;) may become more difficult than E(x, 
due to the inverse of matrix. It indicates that the complexity of solving the two nonlinear 
equations will be distinctive for different problems, which is the original source of why 
we design Strategy 1 and Strategy 2. 

Any way, finding stationary points is just one way to solve optimization problems, and 
we can use iterative method to "search" for global optimum as well. According to the 
results of canonical duality theory, compared with the primal problem (V), the advan- 
tages of solving the canonical dual problem (V d ) is that (V d ) can be easily solved by 
well-developed optimization algorithms because the {—V d ) is convex on convex domain 
Sq. In practice, we find that sometimes the form of (V d ) may become much more com- 
plicated than the primal problem (V) , also due to complexity of computing (?) , which 
is the original source of why we design Strategy 3 and Strategy 4- 

So far, there still exist big issues in practical application. In terms of Strategy 1 and 
Strategy 2, there may exist numerous stationary points, leading it difficult to identify 
which one is in the canonical dual feasible domain, while for Strategy 3, there will be no 
canonical dual feasible solutions, making it impossible to substitute back to get solution 
to the primal problem. If we suppose that there exists a algorithm, once it runs into 
the "neighborhood" of the canonical dual feasible domain, it will never deviate too far 
from the "neighborhood" , then we can start from an initial point in this "neighborhood" 
to finally arrive at the global solution according to the close relationship between the 
solution of dual and that of the primal by the complementary-dual principle, which is 
the kernel of the proposed four strategies. In this case, there is no need to identify which 
stationary point is in the canonical dual feasible domain <S+ and there is also no need 
to solving a constrained optimization problem, because we can just start from a "good" 
initial point to solve the nonlinear equations or to find global optimum based on uncon- 
strained approach. 

The detailed strategies of how to use the canonical duality theory above to find a global 
minimum will be given in the following: 



October 4, 2012 0:4 Engineering Optimization Four'strategies'to'develop'canonical'duar algorithms 
8 Xiaojun Zhou and Chunhua Yang 



Strategy 1 

1: standardization 

Convert the original problem to the standard form of the primal problem discussed in 
the paper, and then the parameters of (V) like a^, A^, b^, c^, Q, f will be well defined. 
2: selection 

Select an appropriate to make sure that G(<;o) >z 0, and then gain the corresponding 

x = G- 1 Mi ? M- 

3: nonlinear equations 

Taking (xo,Co) as initial point, use numerical calculation methods to solve the 
nonlinear equations in (124ft . 



Strategy 2 

1: standardization 

Convert the original problem to the standard form of the primal problem discussed in 
the paper, and then the parameters of (V) like ctfc, A^, bfc, Q, f will be well defined. 
2: selection 

Select an appropriate Co to make sure that G(<;o) h 0. 
3: nonlinear equations 

Taking as initial point, use numerical calculation methods to solve the 
nonlinear equations in (|25p . 



Strategy 3 

1: standardization 

Convert the original problem to the standard form of the primal problem discussed in 
the paper, and then the parameters of (V) like ctk, Ak, b&, c/c, Q, f will be well defined. 
2: selection 

Select an appropriate Co to make sure that G(^o) ^ 0. 
3: numerical optimization 

Taking as an initial point (or some as initial population), using numerical 
optimization algorithms to optimize the dual problem. 



Strategy 4 

1: standardization 

Convert the original problem to the standard form of the primal problem discussed in 
the paper, and then the parameters of (V) like a&, A^, b^, c^, Q, f will be well defined. 
2: selection 

Select an appropriate to make sure that G(<?o) ^ 0, and then gain the corresponding 

x = G- 1 Mi ? (^o). 

3: numerical optimization 

Taking xo as an initial point (or some xo as initial population), using numerical 
optimization algorithms to optimize the primal problem. 



Remark 1. In the numerical optimization step of Strategy 3 and Strategy 4, some 
numerical optimization methods may not be able to search just in the "neighborhood" , 
in this case, a penalty function is suggested to add to (—V d ) or (V) so that G(<;) y in 
the search process. 
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5. Numerical results 

To testify the effectiveness of the strategies, some fourth-order polynomials benchmark 
functions are collected, and we will use the proposed strategies to find the global min- 
imum one by one. In this paper, we implement the strategies in MATLAB R2010b on 
Intel(R) Core(TM) i3-2310M CPU @2.10GHz under Window 7 environment, and fsolve 
and fminunc built in MATLAB are used to solve nonlinear equations and for numerical 
optimization, respectively. 

Example (A special case) 
Considering the following one-dimensional problem 

fo( x ) = \ a (\ ax2 + bx + cf + ^qx 2 - xf. 
we can get the corresponding canonical dual problem easily 



Fo{ '> ~ Cq 2a' 2(q + a,) 



To be more specific, let fix a = 1, a = 1, b = 
special case because x* = (? -1 (<r).F(c) = 
dual functions are given in Fig(TJ 



T, c = —2, q = —2, / = —2, which is a 
1, V?, and the graphs of the primal and 




Figure 1. graphs of the primal function fa(x) and its corresponding dual function Pq (s) 



As can be shown in Fig{TJ there is no critical point in the canonical dual feasible 
domain G(<;) = —2 + ? > 0, that is to say, the semi-definite programming software 
packages will be invalid in this case. For remedy, we can add a small linear perturbation 
to the primal problem, for instance, / = / + A. If A = 0.05, the graphs of the primal 
and dual functions with linear perturbation are given in Figj2j 

Using SeDuMi to solve the modified canonical dual problem, we can get = 2.0166, 
x* = G" 1 (?)F(<;) = -2.0056 and P(x*) = -6.1234. We can find that there exists small 
deviation from the global optimum. 

On the other hand, if we choose to use the proposed Strategy 4, n °t starting from 
the initial point xq = 1 directly but adding a translation xq = 1 + randn (randn is the 
built-in function of the standard normal distribution within MATLAB), we can finally 
arrive at either x* = —2 or x* = —4 precisely. 
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Example 1 (Colville function) 



/i = ioo(x 2 - xfy + (i - Xl y + 9o(x 4 - xiY + (i - x z y 

+10.1((x 2 - l) 2 + (x 4 - l) 2 ) + 19.8(x 2 - l)(x 4 - 1). 
We firstly rewrite it to the standard form, and then we can get ai = 200, a 2 = 180, A\ 



Diag{-2,0,0,0},A 2 = Diag{0, 0, -2, 0}, b x 
0,f = [2,40,2,40] T , and 



0,1,0,0]^, b 2 = [0,0,0,l] J , Cl = c 2 



Q 



(2 \ 
20.2 19.8 
2 

\0 19.8 20.2/ 



,then G(s) 



/2-2<a 
20.2 




V 



\ 
19.8 
2 - 2 ?2 
19.8 20.2/ 



Strategy 1 

The generalized complementary function is 



E(x, q) = (xj - x 2 )?i + (x 2 - x 4 )ft 



400 



360 



<3 



+(x 2 + 10.1x| + x\ + 10.1x 4 + 19.8x 2 x 4 ) - (2xi + 40x 2 + 2x 3 + 40x 4 ) + 42. 

We select <Jo = (ft,<a) = (0.5,0.5) to make sure that G(<Jo) h and the corresponding 
x = (x l ,X2,x 3 ,x 4 ) = G(so) _1 -F(so) = (2.0000,0.9875,2.0000,0.9875) as initial point 
(xo,Co) f° r the nonlinear equations in (|24p . and after 5 iterations with 0.338344 seconds, 
we obtain (x*,0 = (1.0000, 1.0000, 1.0000, 1.0000,0.0000,0.0000) and P(x*) = 0. 

Strategy 2 

The canonical dual function is 

1 o 1 



P a (<;) = 42 



400 



360 



4 



2, 40-?i,2,40 -<a 



/2-2qx \ 
20.2 19.8 
2 - 2 ?2 
\ 19.8 20.2/ 



/ 2 \ 

40 -ft 
2 

V40- ?2 / 



We select the same <?o = (0.5,0.5) for the nonlinear equations in ([25]) . and after 6 
iterations with 0.302655 seconds, we obtain <j* = (0.0000,0.0000). The corresponding 
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x* = G(0 _1 -F(0 = (!> 1, h 1) and P(x*) = 0. 
Strategy 3 

We choose the same £o = (0.5,0.5) as initial point for (—V d ), and we can finally 
arrive at = (0, 0) with 9 iterations and 0.329829 seconds. The corresponding 

x* = G(0 _1 -F(0 = (!> 1> !> 1) and p ( x *) = 0- 
Strategy 4 

We choose the same = (0.5,0.5), and then we use the corresponding 

x = G(<j ) _1 i ? («Jo) = (2.0000,0.9875,2.0000,0.9875) as initial point for P(x), 
and we can finally arrive at x* = (1,1,1,1) and P(x*) = with 26 iterations and 
0.354057 seconds. 

Example 2 (Zettle function) 

h = {x\ + x\-2x x f + 0.25x1. 

The landscape of Zettle function is given in Fig. [3j 




Figure 3. Landscape of Zettle function 

Firstly, we rewrite it to the standard form, and then we can get a = 2, A = 
Diag{2,2},b = [-2,0] T ,c = 0,Q = 0, / = [-0.25, 0] T , and then G(?) = Diag {2^,2^}. 
Strategy 1 

The generalized complementary function is 

1 c 2 
E(x,0 = (-(2x 2 + 2x^)-2x 1 ) ? --^+0.25x 1 . 

We select £o = 0.1 to make sure that G(<Jo) t: and the corresponding xo = 
G(<?o) _1 P(^o) = (—0.2500,0) as initial point (xo,Sq) for the nonlinear equations in (|2"%]1 . 
and after 3 iterations with 0.290396 seconds, we obtain (x*,<j*) = (-0.0299,0,0.1214) 
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and P(x*) = -0.0038. 
Strategy 2 

The canonical dual function is 

pd (c) = _£ _ & ~ °- 25 ) 2 

{ } 4 4^ 

We select the same $o = 0.1 for the nonlinear equations in (|25l) . and after 
4 iterations with 0.290140 seconds, we obtain = 0.1214. The corresponding 
x* = G , (0~ 1 ^( < ?*) = -0.0299 and P(x*) = -0.0038. 

Strategy 3 

We choose the same <Jo = 0.1 as initial point for {—V ), and we can finally 
arrive at q* = 0.1214 with 4 iterations and 0.319423 seconds. The corresponding 
x* = G{q*)- l F{C) = (-0.0299,0) and P(x*) = -0.0038. 

Strategy 4 

We choose the same — 0.1, and then we use the corresponding xo = G(^o) _1 P(co) 
= (—0.2500,0) as initial point for -P(x), and we can finally arrive at x* = (—0.0299,0) 
and P(x*) = -0.0038 with 5 iterations and 0.309056 seconds. 

Example 3 (Styblinski-Tang function) 



/ 3 = -^-16^ + 54 

8=1 

The landscape of Styblinski-Tang function is given in Fig. [H 




Figure 4. Landscape of Styblinski-Tang function 
At first, we rewrite it to the standard form, and then we can get ax = oli = 1,A\ = 
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Diag{2,0},A 2 = Diag{0,2},b! = b 2 = 0, a = c 2 = 0, Q = Diag{-16, -16}, / = 
[-2.5, -2.5] T , and then G?(?) = Diag{-16 + 2ft, -16 + 2 ?2 }. 
Strategy 1 

The generalized complementary function is 

H(x, <;) = x 2 ft + xi? 2 - y " y + g (" 16x ? " 16x i) + 2 -5xi + 2 -5x 2 . 

We select Co = (8.1,8.1) to make sure that G(so) ^ an d the corresponding 
xo = G(^o) _1 -^ 1 (^o) = (—12.5,-12.5) as initial point (xo,Co) for the nonlinear equa- 
tions in (|24p . and after 8 iterations with 0.305630 seconds, we obtain (x*,<j*) = 
(-2.9035,-2.9035,8.4305,8.4305) and P(x*) = -78.3323. 

Strategy 2 

The canonical dual function is 

pdU) = - q l-$-l ( 6.25 (-16 + 2ft) 6.25(-16 + 2 ?2 ) \ 

{ ' 2 2 2 V 256 - 32ft - 32 ?2 + 4ft ?2 256 - 32ft - 32 ?2 + 4ft ?2 J 

We select the same £o = (8.1,8.1) for the nonlinear equations in (|25|). and after 8 
iterations with 0.305489 seconds, we obtain <;* = (8.4305,8.4305). The corresponding 
x* = G(s*)- l F(<?) = (-2.9035,-2.9035) and P(x*) = -78.3323. 

Strategy 3 

We choose the same <Jo = (8.1,8.1) as initial point for (—V d ), and then we can 
finally arrive at (?*) = (8.4305,8.4305) within 7 iterations and 0.320454 seconds. The 
corresponding x* = G(0 _1 -F(0 = (-2.9035,-2.9035) and P(x*) = -78.3323. 

Strategy 4 

We choose the same <?o = (8.1,8.1), and then we use the corresponding 
xo = G(<Jo) _1 -F(<?o) = (—12.5,-12.5) as initial point for P(x), we can finally arrive at 
x* = (-2.9035, -2.9035) and P(x*) = -78.3323 with 10 iterations and 0.323847 seconds. 

Example 4 (Rosenbrock function) 

n-l 

U = J>00(x m - x 2 ) 2 + ( Xi - l) 2 ]. 
i=l 

At first, we rewrite it to the standard form, and then we can get a k = 200, A k = 
-24 A = e k+1 ,c k = Q,Q = Diag{ 2,2,--- ,2 ,0},/ = [2,2,- •• ,2 ,0] T , where fc = 

n— 1 n— 1 

1, 2, • • • , n— 1, //<: G M" x?1 is a diagonal matrix with all zeros except the position (fc, k) hav- 
ing value 1 and e k € R n is a unit vector with all zeros except the position k having value 
1. Then we can obtain G(s) = Q + Y2=\ Qi A k = Diag{2 - 2ft, 2 - 2? 2 , ■ ■ ■ , 2 - 2<? n _i, 0}, 
^to = / " Efc=! ftfefe = [2, 2 - ft, • • • , 2 - fn _ 2 , -^.i]. 

Without much loss of generality, n = 2 is chosen for simple study, and its corresponding 
landscape is plotted in Figj5l in which, the global minimum is located in a long, deep, 
narrow, banana shaped flat valley. 

Strategy 1 
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Figure 5. Landscape of Rosenbrock function in two dimension 



The generalized complementary function is 

3(x, ?) = (x 2 - xf) q - — + x\ - 2xi + 1 

We select <?o = — 1 to make sure that G(<Jo) t: 0. Using the Moore- Penrose pseudoin- 
verse, we can obtain the corresponding xo = G(<Jo) _1 -P 1 (<?o) = (0.5,0). Taking (xo,?o) 
as initial point for the nonlinear equations in (|24p . and after 4 iterations with 0.304920 
seconds, we obtain (x*,?*) = (1,1,0) and P(x*) = 0. 

Strategy 2 

The canonical dual function is 

Due to the singularity of matrix G(q), we can not get a proper form of P d (q); thus it 
becomes difficult to solve the nonlinear equations in (|25p . 
Strategy 3 

The same situation happens as above, we can choose some possible initial point % to 
guarantee G(%) >: 0, but the calculation of the singular matrix is quite complicated. 
Strategy 4 

Instead, we choose the same ?o> an d using the Moore-Penrose pseudoinverse, we can 
obtain the corresponding xo = (0.5,0) for -P(x). Taking xo as initial point for -P(x), we 
can finally reach x* = (1, 1) within 20 iterations and 0.352472 seconds, and then the 
corresponding P(l, 1) = 0. 

Furthermore, we continue to consider the Rosenbrock function in terms of large 
dimensions. We choose = ( — 1> • • • ) — 1) to guarantee G(<?o) h 0, and then get the 
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corresponding initial point xo = (0.5,0.75, • • • ,0.75,0) for P(x). General results of the 

V v ' 

n-2 

Rosenbrock function by Strategy 4 are given in Table [TJ 



Table 1. Results of the Rosenbrock function using Strategy 4 



11 


X* 




P(x*) 


iterations 


time(s) 


2 


(1,1 


) 


2.0269e-011 


20 


0.352472 


5 


(1,- 


,1) 


5.4958e-011 


29 


0.405747 


10 


(1,... 


•1) 


1.0633e-010 


31 


0.409724 


20 


(1,.. 


•1) 


5.3688e-011 


37 


0.423663 


50 


(1,.. 


•1) 


1.6986e-009 


42 


0.554678 


100 


(1,.. 


•1) 


3.7337e-010 


50 


0.727062 


200 


(1,.. 


•1) 


1.5632e-010 


55 


1.329283 


500 


(1,.. 


•1) 


3.0872e-010 


54 


3.508815 


1000 


(1,.. 


•1) 


5.0893e-010 


56 


8.763668 


2000 


(1,.. 


•1) 


3.7200e-010 


60 


28.264277 


3000 


(1,.. 


•1) 


7.3433e-010 


62 


57.669020 


4000 


(1,.. 


•1) 


1.0350e-009 


61 


92.344600 


5000 


(1,.. 


,1) 


1.0340e-009 


66 


144.069188 



Compared the results for the Rosenbrock functi on with thos e gain e d by most popula r 
stochastic meth ods, like PSO (CLPSO, APSOlrtLiang et all 120061 . IZhan et all 120091 ) 
and DE fSaDEl riQin et al.ll2009l . iDas and Sue anthanll20lin 7 we can conclude that the 
strategy used in this paper by canonical duality theory is much more superior. To the 
best of our knowledge, it is the first time to solve the Rosenbrock function optimization 
problem up to 5000 dimension in such a short time. 



Example 5 (Dixon and Price function) 

n 

/5 = (x 1 -l) 2 + ^i(2^ 2 -X i _ 1 ) 2 . 
i=2 



The landscape of Dixon and Price function is given in Fig. [6l 

We firstly rewrite it to the standard form, and then we can get a k = 2{k + 1), 
A k = 4/ fc+ i, h = -e k , c k = 0, Q = Diag{2, 0,-- ,0 }, / = [2,0, • ■■ ^0] T , where 

n— 1 n— 1 

k = 1,2, ■■■ ,n - 1, and then G(<?) = Q + YZ=\ ^k^-h = Diag{2, 4cl, • ■ ■ ,4? n _i}, 

F(S) = f~ ElZl Skb k = [2 + ?i, ? 2 , • • • , ?n-i, 0]. 

It is not difficult to find that, for any dual feasible solution <j*, if we substitute 
back then we will find that the last component of the corresponding x* will always be 
zero, which indicates that the first three strategies will be invalid. However, the fourth 
strategy can still survive if we make some minor revisions. For simplicity, we choose 
= (2, • • ■ ,2) to make sure G(^o) h 0, and then get the corresponding initial point 
xo = (2, 0.25, • • • , 0.25, 0) for -P(x). We don't use the xq directly but translate the point 



to xo = xo + 1 = (3, 1.25, ■ ■ ■ , 1.25, 1). Taking the revised initial point for the primal 

V v ' 

n-2 
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Figure 6. Landscape of Dixon and Price function in two dimension 



problem, the general results of the Dixon and Price function are given in Table [2j 



Table 2. Results of the Dixon and Price function using Strategy 4 



11 




X* 


P(x*) 


iterations 


time(s) 


2 


(1 


,0.7071) 


3.1388e-015 


12 


0.213785 


5 


(V 


• ,0.5221) 


8.4890e-014 


21 


0.206739 


10 


(V 


• ,0.5007) 


5.4620e-012 


30 


0.218370 


20 


(1,- 


• ,0.5000) 


9.1666e-011 


46 


0.245217 


50 


(1,- 


• ,0.5000) 


3.4299e-010 


79 


0.388959 


100 


(1,- 


• ,0.5000) 


3.6424e-009 


108 


0.757873 


200 


(1,- 


• ,0.5000) 


1.0303e-008 


154 


1.720907 


500 


(1,- 


• ,0.5000) 


3.1588e-008 


242 


7.814894 


1000 


(1,- 


• ,0.5000) 


6.8696e-008 


342 


28.862242 


2000 


(1,- 


• ,0.5000) 


1.3657e-007 


480 


124.977932 


3000 


(1,- 


• ,0.5000) 


2.4159e-007 


581 


270.350883 


4000 


(1,- 


• ,0.5000) 


2.2758e-007 


675 


526.158263 


5000 


(1,- 


• ,0.5000) 


3.5225e-007 


747 


854.212220 



6. Conclusion 



To efficiently apply the canonical duality theory for real world problems, four strategies 
are proposed to develop algorithms based on the theory. The former two strategies should 
calculate the staionary points, in other words, solving nonlinear equations, while the later 
strategies use numerical optimization algorithms based on unconstrained methods. Some 
experimental results are given to illustrate the details of using the four strategies for 
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fourth-order polynomial benchmark functions, and we find that various strategies have 
different degrees of complexity. To some extent, the canonical duality theory can eliminate 
the gap between deterministic and stochastic methods. In our future work, we will try to 
use stochastic methods to design efficient algorithms for the powerful canonical duality 
theory. 
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